Camera 使用
1. Camera 简介
ArmSoM系列产品使用的是mipi-csi接口的摄像头
ArmSoM-Sige7支持双摄同显：
2. RK3588硬件通路框图
rk3588支持2个isp硬件，每个isp设备可虚拟出多个虚拟节点，软件上通过回读的方式，依次从ddr读取每一路的图像数据进isp处理。对于多摄方案，建议将数据流平均分配到两个isp上。
回读：指数据经过vicap采集到ddr，应用获取到数据后，将buffer地址推送给isp，isp再从ddr获取图像数据。
3. RK3588 的camera通路:
多sensor支持：
- 单路硬件isp最多支持4路复用，isp复用情况支持分辨率如下：
- 2路复用：最大分辨率3840x2160，dts对应配置2路rkisp_vir设备。
- 3路或4路复用：最大分辨率2560x1536，dts对应配置3或4路rkisp_vir设备。
- 硬件支持最多采集7路sensor：6mipi + 1dvp，多sensor软件通路如下：
下图是RK3588 camera连接链路示意图，可以支持7路camera。
4. 链路解析：
- 图中：mipi camera2---> csi2_dphy1 ---> mipi2_csi2 ---> rkcif_mipi_lvds2--->rkcif_mipi_lvds2_sditf --->rkisp0_vir2
- 对应节点：imx415 ---> csi2_dphy0 ---> mipi2_csi2 ---> rkcif_mipi_lvds2--->rkcif_mipi_lvds2_sditf --->rkisp0_vir2
- 链接关系：sensor---> csi2 dphy---->mipi csi host--->vicap
- 实线链路解析： Camera sensor ---> dphy ---> 通过mipi_csi2模块解析mipi协议---> vicap ( rkcif节点代表vicap )
- 虚线链路解析：vicap ---> rkcif_mipi_lvds2_sditf ---> isp
提示
每个vicap节点与isp的链接关系，通过对应虚拟出的XXX_sditf来指明链接关系。
5. ArmSoM-Sige7双路Camera调试
这里以imx415摄像头为例，解析ArmSoM-Sige7双路Camera调试
5.1 原理图
Camera1：CSI0_MIPI：
Camera2： CS1_MIPI：
5.2 双路Camera的dts配置：
- 链路配置1： imx415 —> csi2_dphy0 —> mipi2_csi2 —> rkcif_mipi_lvds2—>rkcif_mipi_lvds2_sditf —>rkisp0_vir2
- 链路配置2： imx415 —> csi2_dphy3 —> mipi4_csi2 —> rkcif_mipi_lvds4—>rkcif_mipi_lvds4_sditf —>rkisp1_vir1
&i2c3 {
status = "okay";
imx415: imx415@1a {
status = "okay";
compatible = "sony,imx415";
reg = <0x1a>;
clocks = <&cru CLK_MIPI_CAMARAOUT_M3>;
clock-names = "xvclk";
pinctrl-names = "default";
pinctrl-0 = <&mipim0_camera3_clk>;
power-domains = <&power RK3588_PD_VI>;
pwdn-gpios = <&gpio1 RK_PB0 GPIO_ACTIVE_HIGH>;
reset-gpios = <&gpio4 RK_PA0 GPIO_ACTIVE_LOW>;
rockchip,camera-module-index = <0>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "CMK-OT2022-PX1";
rockchip,camera-module-lens-name = "IR0147-50IRC-8M-F20";
port {
imx415_out0: endpoint {
remote-endpoint = <&mipidphy0_in_ucam0>;
data-lanes = <1 2 3 4>;
};
};
};
};
&i2c4 {
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <&i2c4m1_xfer>;
imx415_1: imx415_1@1a {
status = "okay";
compatible = "sony,imx415";
reg = <0x1a>;
clocks = <&cru CLK_MIPI_CAMARAOUT_M2>;
clock-names = "xvclk";
pinctrl-names = "default";
pinctrl-0 = <&mipim0_camera2_clk>;
power-domains = <&power RK3588_PD_VI>;
pwdn-gpios = <&gpio3 RK_PA2 GPIO_ACTIVE_HIGH>;
reset-gpios = <&gpio3 RK_PA0 GPIO_ACTIVE_LOW>;
rockchip,camera-module-index = <1>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "CMK-OT2022-PX1";
rockchip,camera-module-lens-name = "IR0147-50IRC-8M-F20";
port {
imx415_out3: endpoint {
remote-endpoint = <&mipidphy3_in_ucam3>;
data-lanes = <1 2 3 4>;
};
};
};
};
&csi2_dphy0_hw {
status = "okay";
};
&csi2_dphy1_hw {
status = "okay";
};
&csi2_dphy0 {
status = "okay";
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
mipidphy0_in_ucam0: endpoint@1 {
reg = <1>;
remote-endpoint = <&imx415_out0>;
data-lanes = <1 2 3 4>;
};
};
port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
csidphy0_out: endpoint@0 {
reg = <0>;
remote-endpoint = <&mipi2_csi2_input>;
};
};
};
};
&csi2_dphy3 {
status = "okay";
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
mipidphy3_in_ucam3: endpoint@1 {
reg = <1>;
remote-endpoint = <&imx415_out3>;
data-lanes = <1 2 3 4>;
};
};
port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
csidphy3_out: endpoint@0 {
reg = <0>;
remote-endpoint = <&mipi4_csi2_input>;
};
};
};
};
&mipi2_csi2 {
status = "okay";
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
mipi2_csi2_input: endpoint@1 {
reg = <1>;
remote-endpoint = <&csidphy0_out>;
};
};
port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
mipi2_csi2_output: endpoint@0 {
reg = <0>;
remote-endpoint = <&cif_mipi2_in0>;
};
};
};
};
&mipi4_csi2 {
status = "okay";
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
mipi4_csi2_input: endpoint@1 {
reg = <1>;
remote-endpoint = <&csidphy3_out>;
};
};
port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
mipi4_csi2_output: endpoint@0 {
reg = <0>;
remote-endpoint = <&cif_mipi_in4>;
};
};
};
};
&rkcif {
status = "okay";
};
&rkcif_mipi_lvds2 {
status = "okay";
port {
cif_mipi2_in0: endpoint {
remote-endpoint = <&mipi2_csi2_output>;
};
};
};
&rkcif_mipi_lvds2_sditf {
status = "okay";
port {
mipi_lvds2_sditf: endpoint {
remote-endpoint = <&isp0_vir2>;
};
};
};
&rkcif_mipi_lvds4 {
status = "okay";
port {
cif_mipi_in4: endpoint {
remote-endpoint = <&mipi4_csi2_output>;
};
};
};
&rkcif_mipi_lvds4_sditf {
status = "okay";
port {
mipi4_lvds_sditf: endpoint {
remote-endpoint = <&isp1_vir1>;
};
};
};
&rkcif_mmu {
status = "okay";
};
&rkisp0 {
status = "okay";
};
&isp0_mmu {
status = "okay";
};
&rkisp0_vir2 {
status = "okay";
port {
#address-cells = <1>;
#size-cells = <0>;
isp0_vir2: endpoint@0 {
reg = <0>;
remote-endpoint = <&mipi_lvds2_sditf>;
};
};
};
&rkisp1 {
status = "okay";
};
&isp1_mmu {
status = "okay";
};
&rkisp1_vir1 {
status = "okay";
port {
#address-cells = <1>;
#size-cells = <0>;
isp1_vir1: endpoint@0 {
reg = <0>;
remote-endpoint = <&mipi4_lvds_sditf>;
};
};
};
&pinctrl {
camera {
cam_pwdn_gpio: cam-pwdn-gpio {
rockchip,pins = <1 RK_PB0 RK_FUNC_GPIO &pcfg_pull_up>;
};
};
};
6. 调试技巧
6.1 查看Camera是否挂载到i2c总线下
i2cdetect -y 3
6.2 查看拓扑结构
media-ctl -d /dev/media0 -p
6.3 imx415 相关的log信息
dmesg | grep imx415
6.4 查看sys文件系统中文件信息
内核会为摄像头在目录/sys/class/video4linux下分配设备信息描述文件
armsom@armsom:~$ grep imx415 /sys/class/video4linux/v*/name
/sys/class/video4linux/v4l-subdev2/name:m00_b_imx415 3-001a
/sys/class/video4linux/v4l-subdev7/name:m01_b_imx415 4-001a
查找Camera对应的vedio节点：
armsom@armsom:~$ grep "" /sys/class/video4linux/v*/name | grep mainpath
/sys/class/video4linux/video22/name:rkisp_mainpath
/sys/class/video4linux/video31/name:rkisp_mainpath
可以看到，在ArmSoM-Sige7中，双Camera的节点对应的是：video22和video31
6.5 查找所有摄像头设备
armsom@armsom:~$ v4l2-ctl --list-devices
rkisp-statistics (platform: rkisp):
/dev/video29
/dev/video30
/dev/video38
/dev/video39
rkcif-mipi-lvds2 (platform:rkcif):
/dev/media0
/dev/media1
rkcif (platform:rkcif-mipi-lvds2):
/dev/video0
/dev/video1
/dev/video2
/dev/video3
/dev/video4
/dev/video5
/dev/video6
/dev/video7
/dev/video8
/dev/video9
/dev/video10
rkcif (platform:rkcif-mipi-lvds4):
/dev/video11
/dev/video12
/dev/video13
/dev/video14
/dev/video15
/dev/video16
/dev/video17
/dev/video18
/dev/video19
/dev/video20
/dev/video21
rkisp_mainpath (platform:rkisp0-vir0):
/dev/video22
/dev/video23
/dev/video24
/dev/video25
/dev/video26
/dev/video27
/dev/video28
/dev/media2
rkisp_mainpath (platform:rkisp1-vir1):
/dev/video31
/dev/video32
/dev/video33
/dev/video34
/dev/video35
/dev/video36
/dev/video37
/dev/media3
其中/dev/video22和/dev/video31都是摄像头的设备。
6.6 查看设备的预览支持格式
如下是video22节点： imx415 摄像头的查询结果：
armsom@armsom:~$ v4l2-ctl -d /dev/video22 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar
[0]: 'UYVY' (UYVY 4:2:2)
Size: Stepwise 32x32 - 3840x2160 with step 8/8
[1]: 'NV16' (Y/CbCr 4:2:2)
Size: Stepwise 32x32 - 3840x2160 with step 8/8
[2]: 'NV61' (Y/CrCb 4:2:2)
Size: Stepwise 32x32 - 3840x2160 with step 8/8
[3]: 'NV21' (Y/CrCb 4:2:0)
Size: Stepwise 32x32 - 3840x2160 with step 8/8
[4]: 'NV12' (Y/CbCr 4:2:0)
Size: Stepwise 32x32 - 3840x2160 with step 8/8
[5]: 'NM21' (Y/CrCb 4:2:0 (N-C))
Size: Stepwise 32x32 - 3840x2160 with step 8/8
[6]: 'NM12' (Y/CbCr 4:2:0 (N-C))
Size: Stepwise 32x32 - 3840x2160 with step 8/8
6.7 查看设备的所有信息：
armsom@armsom:~$ v4l2-ctl --all --device /dev/video22
Driver Info:
Driver name : rkisp_v6
Card type : rkisp_mainpath
Bus info : platform:rkisp0-vir0
Driver version : 2.3.0
Capabilities : 0x84201000
Video Capture Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04201000
Video Capture Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : rkisp0-vir0
Model : rkisp0
Serial :
Bus info :
Media version : 5.10.160
Hardware revision: 0x00000000 (0)
Driver version : 5.10.160
Interface Info:
ID : 0x03000007
Type : V4L Video
Entity Info:
ID : 0x00000006 (6)
Name : rkisp_mainpath
Function : V4L2 I/O
Pad 0x01000009 : 0: Sink
Link 0x0200000a: from remote pad 0x1000004 of entity 'rkisp-isp-subdev' (Unknown V4L2 Sub-Device): Data, Enabled
Priority: 2
Format Video Capture Multiplanar:
Width/Height : 3840/2160
Pixel Format : 'NM12' (Y/CbCr 4:2:0 (N-C))
Field : None
Number of planes : 2
Flags :
Colorspace : sRGB
Transfer Function : Rec. 709
YCbCr/HSV Encoding: Rec. 709
Quantization : Full Range
Plane 0 :
Bytes per Line : 3840
Size Image : 8294400
Plane 1 :
Bytes per Line : 3840
Size Image : 4147200
Selection Video Capture: crop, Left 0, Top 0, Width 3840, Height 2160, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 3840, Height 2160, Flags:
Selection Video Output: crop, Left 0, Top 0, Width 3840, Height 2160, Flags:
Selection Video Output: crop_bounds, Left 0, Top 0, Width 3840, Height 2160, Flags:
Image Processing Controls
pixel_rate 0x009f0902 (int64) : min=0 max=1000000000 step=1 default=1000000000 value=356800000 flags=read-only, volatile
6.8 摄像头预览
ArmSoM-Sige7中，双Camera的预览命令：
- 预览摄像头1：
gst-launch-1.0 v4l2src device=/dev/video22 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! autovideosink
- 预览摄像头2：
gst-launch-1.0 v4l2src device=/dev/video31 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! autovideosink
7. Camera应用程序开发
客户可以根据自己的需求进行Camera相关的应用程序开发，如下是使用QT开发的双摄同显应用程序：
